Universal Disk Format
   HOME

TheInfoList



OR:

Universal Disk Format (UDF) is an
open Open or OPEN may refer to: Music * Open (band), Australian pop/rock band * The Open (band), English indie rock band * ''Open'' (Blues Image album), 1969 * ''Open'' (Gotthard album), 1999 * ''Open'' (Cowboy Junkies album), 2001 * ''Open'' (YF ...
, vendor-neutral
file system In computing, file system or filesystem (often abbreviated to fs) is a method and data structure that the operating system uses to control how data is stored and retrieved. Without a file system, data placed in a storage medium would be one larg ...
for
computer data storage Computer data storage is a technology consisting of computer components and Data storage, recording media that are used to retain digital data (computing), data. It is a core function and fundamental component of computers. The central pro ...
for a broad range of media. In practice, it has been most widely used for
DVD The DVD (common abbreviation for Digital Video Disc or Digital Versatile Disc) is a digital optical disc data storage format. It was invented and developed in 1995 and first released on November 1, 1996, in Japan. The medium can store any kind ...
s and newer
optical disc In computing and optical disc recording technologies, an optical disc (OD) is a flat, usually circular disc that encodes binary data (bits) in the form of pits and lands on a special material, often aluminum, on one of its flat surfaces. ...
formats, supplanting
ISO 9660 ISO 9660 (also known as ECMA-119) is a file system for optical disc media. Being sold by the International Organization for Standardization (ISO) the file system is considered an international technical standard. Since the specification is av ...
. Due to its design, it is very well suited to incremental updates on both recordable and (re)writable
optical media In computing and optical disc recording technologies, an optical disc (OD) is a flat, usually circular disc that encodes binary data ( bits) in the form of pits and lands on a special material, often aluminum, on one of its flat surfaces ...
. UDF was developed and maintained by the
Optical Storage Technology Association The Optical Storage Technology Association (OSTA) was an international trade association formed to promote the use of recordable optical data storage technologies and products. It was responsible for the creation and maintenance of the Universal ...
(OSTA). In engineering terms, Universal Disk Format is a
profile Profile or profiles may refer to: Art, entertainment and media Music * ''Profile'' (Jan Akkerman album), 1973 * ''Profile'' (Githead album), 2005 * ''Profile'' (Pat Donohue album), 2005 * ''Profile'' (Duke Pearson album), 1959 * '' ''Profi ...
of the specification known as ISO/IEC 13346 and ECMA-167.


Usage

Normally, authoring software will master a UDF file system in a batch process and write it to optical media in a single pass. But when
packet writing Packet writing (or incremental packet writing, IPW) is an optical disc recording technology used to allow write-once and rewritable CD and DVD media to be used in a similar manner to a floppy disk from within the operating system. Details ...
to rewritable media, such as
CD-RW CD-RW (Compact Disc-Rewritable) is a digital media, digital optical disc data storage device, storage format introduced in 1997. A CD-RW compact disc (CD-RWs) can be written, read, erased, and re-written. CD-RWs, as opposed to CDs, require ...
, UDF allows files to be created, deleted and changed on-disc just as a general-purpose
filesystem In computing, file system or filesystem (often abbreviated to fs) is a method and data structure that the operating system uses to control how data is Computer data storage, stored and retrieved. Without a file system, data placed in a storage me ...
would on removable media like
floppy disk A floppy disk or floppy diskette (casually referred to as a floppy, or a diskette) is an obsolescent type of disk storage composed of a thin and flexible disk of a magnetic storage medium in a square or nearly square plastic enclosure lined w ...
s and
flash drive A flash drive is a portable computer drive that uses flash memory. Flash drives are the larger memory modules consisting of a number of flash chips. A flash chip is used to read the contents of a single cell, but it can write entire block of cell ...
s. This is also possible on write-once media, such as
CD-R CD-R (Compact disc-recordable) is a digital optical disc storage format. A CD-R disc is a compact disc that can be written once and read arbitrarily many times. CD-R discs (CD-Rs) are readable by most CD readers manufactured prior to the i ...
, but in that case the space occupied by the deleted files cannot be reclaimed (and instead becomes inaccessible). Multi-session mastering is also possible in UDF, though some implementations may be unable to read disks with multiple sessions.


History

The
Optical Storage Technology Association The Optical Storage Technology Association (OSTA) was an international trade association formed to promote the use of recordable optical data storage technologies and products. It was responsible for the creation and maintenance of the Universal ...
standardized the UDF file system to form a common file system for all optical media: both for read-only media and for re-writable optical media. When first standardized, the UDF file system aimed to replace
ISO 9660 ISO 9660 (also known as ECMA-119) is a file system for optical disc media. Being sold by the International Organization for Standardization (ISO) the file system is considered an international technical standard. Since the specification is av ...
, allowing support for both read-only and writable media. After the release of the first version of UDF, the
DVD Consortium The DVD Forum is an international organization composed of hardware, software, media and production companies that use and develop the DVD and formerly HD DVD formats. It was initially known as the DVD Consortium when it was founded in 1995. H ...
adopted it as the official file system for
DVD-Video DVD-Video is a consumer video format used to store digital video on DVD discs. DVD-Video was the dominant consumer home video format in Asia, North America, Europe, and Australia in the 2000s until it was supplanted by the high-definition Blu-r ...
and
DVD-Audio DVD-Audio (commonly abbreviated as DVD-A) is a digital format for delivering high-fidelity audio content on a DVD. DVD-Audio uses most of the storage on the disc for high-quality audio and is not intended to be a video delivery format. The sta ...
. UDF shares the basic volume descriptor format with ISO 9660. A "UDF Bridge" format is defined since 1.50 so that a disc can also contain a ISO 9660 file system making references to files on the UDF part.


Revisions

Multiple revisions of UDF have been released: * Revision 1.00Mentioned only in history of Revision 1.02 (24 October 1995). Original Release. * Revision 1.01 (3 November 1995). Added DVD Appendix and made a few minor changes. * Revision 1.02 (30 August 1996). This format is used by
DVD-Video DVD-Video is a consumer video format used to store digital video on DVD discs. DVD-Video was the dominant consumer home video format in Asia, North America, Europe, and Australia in the 2000s until it was supplanted by the high-definition Blu-r ...
discs. * Revision 1.50 (4 February 1997). Added support for (virtual) rewritability on CD-R/DVD-R media by introducing the VAT structure. Added sparing tables for defect management on rewritable media such as CD-RW, and DVD-RW and DVD+RW. Add UDF bridge. * Revision 2.00 (3 April 1998). Added support for Stream Files and real-time files (for
DVD recording A DVD recorder is an optical disc recorder that uses optical disc recording technologies to digitally record analog or digital signals onto blank writable DVD media. Such devices are available as either installable drives for computers or ...
) and simplified directory management. VAT support was extended. * Revision 2.01 (15 March 2000) is mainly a bugfix release to UDF 2.00. Many of the UDF standard's ambiguities were resolved in version 2.01. * Revision 2.50 (30 April 2003). Added the Metadata Partition facilitating metadata clustering, easier crash recovery and optional duplication of file system information: All metadata like nodes and directory contents are written on a separate partition which can optionally be mirrored. This format is used by some versions of Blu-rays and most HD-DVD discs. * Revision 2.60 (1 March 2005). Added Pseudo OverWrite method for drives supporting pseudo overwrite capability on sequentially recordable media. Has read-only compatibility with UDF 2.50 implementations. (Some
Blu-ray The Blu-ray Disc (BD), often known simply as Blu-ray, is a digital optical disc data storage format. It was invented and developed in 2005 and released on June 20, 2006 worldwide. It is designed to supersede the DVD format, and capable of sto ...
s use this format.) UDF Revisions are internally encoded as binary-coded decimals; Revision 2.60, for example, is represented as . In addition to declaring its own revision, compatibility for each volume is defined by the minimum read and minimum write revisions, each signalling the requirements for these operations to be possible for every structure on this image. A "maximum write" revision additionally records the highest UDF support level of all the implementations that has written to this image. For example, a UDF 2.01 volume that does not use Stream Files (introduced in UDF 2.00) but uses VAT (UDF 1.50) created by a UDF 2.60-capable implementation may have the revision declared as , the minimum read revision set to , the minimum write to , and the maximum write to .


Specifications

The UDF standard defines three
file system In computing, file system or filesystem (often abbreviated to fs) is a method and data structure that the operating system uses to control how data is stored and retrieved. Without a file system, data placed in a storage medium would be one larg ...
variations, called "builds". These are: * ''Plain'' (Random Read/Write Access). This is the original format supported in all UDF revisions * ''Virtual Allocation Table a.k.a. VAT'' (Incremental Writing). Used specifically for writing to write-once media * ''Spared'' (Limited Random Write Access). Used specifically for writing to rewritable media


Plain build

Introduced in the first version of the standard, this format can be used on any type of disk that allows random read/write access, such as
hard disk A hard disk drive (HDD), hard disk, hard drive, or fixed disk is an electro-mechanical data storage device that stores and retrieves digital data using magnetic storage with one or more rigid rapidly rotating platters coated with magnet ...
s,
DVD+RW DVD recordable and DVD rewritable are optical disc recording technologies. Both terms describe DVD optical discs that can be written to by a DVD recorder, whereas only 'rewritable' discs are able to erase and rewrite data. Data is written ('burn ...
and
DVD-RAM DVD-RAM (DVD Random Access Memory) is a DVD-based disc specification presented in 1996 by the DVD Forum, which specifies rewritable DVD-RAM media and the appropriate DVD writers. DVD-RAM media have been used in computers as well as camcorders ...
media. Metadata (up to v2.50) and file data is addressed more or less directly. In writing to such a disk in this format, any physical block on the disk may be chosen for allocation of new or updated files. Since this is the ''basic'' format, practically any operating system or file system driver claiming support for UDF should be able to read this format.


VAT build

Write-once media such as
DVD-R DVD recordable and DVD rewritable are optical disc recording technologies. Both terms describe DVD optical discs that can be written to by a DVD recorder, whereas only 'rewritable' discs are able to erase and rewrite data. Data is written ('burne ...
and
CD-R CD-R (Compact disc-recordable) is a digital optical disc storage format. A CD-R disc is a compact disc that can be written once and read arbitrarily many times. CD-R discs (CD-Rs) are readable by most CD readers manufactured prior to the i ...
have limitations when being written to, in that each physical block can only be written to once, and the writing must happen incrementally. Thus the ''plain'' build of UDF can only be written to CD-Rs by pre-mastering the data and then writing all data in one piece to the media, similar to the way an
ISO 9660 ISO 9660 (also known as ECMA-119) is a file system for optical disc media. Being sold by the International Organization for Standardization (ISO) the file system is considered an international technical standard. Since the specification is av ...
file system gets written to CD media. To enable a CD-R to be used virtually like a hard disk, whereby the user can add and modify files on a CD-R at will (so-called "drive letter access" on Windows), OSTA added the ''VAT'' build to the UDF standard in its revision 1.5. The VAT is an additional structure on the disc that allows
packet writing Packet writing (or incremental packet writing, IPW) is an optical disc recording technology used to allow write-once and rewritable CD and DVD media to be used in a similar manner to a floppy disk from within the operating system. Details ...
; that is, remapping physical blocks when files or other data on the disc are modified or deleted. For write-once media, the entire disc is virtualized, making the write-once nature transparent for the user; the disc can be treated the same way one would treat a rewritable disc. The write-once nature of CD-R or DVD-R media means that when a file is deleted on the disc, the file's data still remains on the disc. It does not appear in the directory any more, but it still occupies the original space where it was stored. Eventually, after using this scheme for some time, the disc will be full, as free space cannot be recovered by deleting files. Special tools can be used to access the previous state of the disc (the state before the delete occurred), making recovery possible. Not all drives fully implement version 1.5 or higher of the UDF, and some may therefore be unable to handle VAT builds.


Spared (RW) build

Rewriteable media such as
DVD-RW DVD recordable and DVD rewritable are optical disc recording technologies. Both terms describe DVD optical discs that can be written to by a DVD recorder, whereas only 'rewritable' discs are able to erase and rewrite data. Data is written ('burne ...
and
CD-RW CD-RW (Compact Disc-Rewritable) is a digital media, digital optical disc data storage device, storage format introduced in 1997. A CD-RW compact disc (CD-RWs) can be written, read, erased, and re-written. CD-RWs, as opposed to CDs, require ...
have fewer limitations than DVD-R and CD-R media. Sectors can be rewritten at random (though in packets at a time). These media can be erased entirely at any time, making the disc blank again, ready for writing a new UDF or other file system (e.g., ISO 9660 or CD Audio) to it. However, sectors of ''-RW'' media may "wear out" after a while, meaning that their data becomes unreliable, through having been rewritten too often (typically after a few hundred rewrites, with CD-RW). The plain and VAT builds of the UDF format can be used on rewriteable media, with some limitations. If the plain build is used on a -RW media, file-system level modification of the data must not be allowed, as this would quickly wear out often-used sectors on the disc (such as those for directory and block allocation data), which would then go unnoticed and lead to data loss. To allow modification of files on the disc, rewriteable discs can be used like -R media using the ''VAT'' build. This ensures that all blocks get written only once (successively), ensuring that there are no blocks that get rewritten more often than others. This way, a RW disc can be erased and reused many times before it should become unreliable. However, it will eventually become unreliable with no easy way of detecting it. When using the ''VAT'' build, CD-RW/DVD-RW media effectively appears as CD-R or DVD+/-R media to the computer. However, the media may be erased again at any time. The ''spared'' build was added in revision 1.5 to address the particularities of rewriteable media. This build adds an extra ''Sparing Table'' in order to manage the defects that will eventually occur on parts of the disc that have been rewritten too many times. This table keeps track of worn-out sectors and remaps them to working ones. UDF defect management does not apply to systems that already implement another form of defect management, such as
Mount Rainier Mount Rainier (), indigenously known as Tahoma, Tacoma, Tacobet, or təqʷubəʔ, is a large active stratovolcano in the Cascade Range of the Pacific Northwest, located in Mount Rainier National Park about south-southeast of Seattle. With a s ...
(MRW) for optical discs, or a disk controller for a hard drive. The tools and drives that do not fully support revision 1.5 of UDF will ignore the sparing table, which would lead them to read the outdated worn-out sectors, leading to retrieval of corrupted data. The so-called UDF ''overhead'' that is spread over the entire disc reserves a portion of the data storage space, limiting the useable capacity of
CD-RW CD-RW (Compact Disc-Rewritable) is a digital media, digital optical disc data storage device, storage format introduced in 1997. A CD-RW compact disc (CD-RWs) can be written, read, erased, and re-written. CD-RWs, as opposed to CDs, require ...
with e.g. 650 MB of original capacity to around 500 MB.


Character set

The UDF specifications allow only one Character Set ''OSTA CS0'', which can store any
Unicode Unicode, formally The Unicode Standard,The formal version reference is is an information technology Technical standard, standard for the consistent character encoding, encoding, representation, and handling of Character (computing), text expre ...
Code point In character encoding terminology, a code point, codepoint or code position is a numerical value that maps to a specific character. Code points usually represent a single grapheme—usually a letter, digit, punctuation mark, or whitespace—but ...
excluding U+FEFF and U+FFFE. Additional character sets defined in ECMA-167 are not used. Since Errata DCN-5157, the range of code points was expanded to all code points from Unicode 4.0 (or any newer or older version), which includes
Plane Plane(s) most often refers to: * Aero- or airplane, a powered, fixed-wing aircraft * Plane (geometry), a flat, 2-dimensional surface Plane or planes may also refer to: Biology * Plane (tree) or ''Platanus'', wetland native plant * ''Planes' ...
1-16 characters such as
Emoji An emoji ( ; plural emoji or emojis) is a pictogram, logogram, ideogram or smiley embedded in text and used in electronic messages and web pages. The primary function of emoji is to fill in emotional cues otherwise missing from typed conversat ...
. DCN-5157 also recommends normalizing the strings to Normalization Form C. The OSTA CS0 character set stores a 16-bit Unicode string "compressed" into 8-bit or 16-bit units, preceded by a single-byte "compID" tag to indicate the compression type. The 8-bit storage is functionally equivalent to ISO-8859-1, and the 16-bit storage is
UTF-16 UTF-16 (16-bit computing, 16-bit Unicode Transformation Format) is a character encoding capable of encoding all 1,112,064 valid code points of Unicode (in fact this number of code points is dictated by the design of UTF-16). The encoding is variab ...
in big endian. The reference algorithm neither checks for forbidden code points nor interprets surrogate pairs, so like
NTFS New Technology File System (NTFS) is a proprietary journaling file system developed by Microsoft. Starting with Windows NT 3.1, it is the default file system of the Windows NT family. It superseded File Allocation Table (FAT) as the preferred fil ...
the string may be malformed. (No specific form of storage is specified by DCN-5157, but UTF-16BE is the only well-known method for storing all of Unicode while being mostly backward compatible with
UCS-2 The Universal Coded Character Set (UCS, Unicode) is a standard set of characters defined by the international standard ISO/IEC 10646, ''Information technology — Universal Coded Character Set (UCS)'' (plus amendments to that standard), whi ...
.)


Compatibility

Many
DVD player A DVD player is a device that plays DVDs produced under both the DVD-Video and DVD-Audio technical standards, two different and incompatible standards. Some DVD players will also play audio CDs. DVD players are connected to a television to ...
s do not support any UDF revision other than version 1.02. Discs created with a newer revision may still work in these players if the ISO 9660 bridge format is used. Even if an
operating system An operating system (OS) is system software that manages computer hardware, software resources, and provides common services for computer programs. Time-sharing operating systems schedule tasks for efficient use of the system and may also in ...
claims to be able to read UDF 1.50, it still may only support the ''plain'' build and not necessarily either the ''VAT'' or ''Spared'' UDF builds.
Mac OS X macOS (; previously OS X and originally Mac OS X) is a Unix operating system developed and marketed by Apple Inc. since 2001. It is the primary operating system for Apple's Mac (computer), Mac computers. Within the market of ...
10.4.5 claims to support Revision 1.50 (see ''man mount_udf''), yet it can only mount disks of the ''plain'' build properly and provides no virtualization support at all. It cannot mount UDF disks with VAT, as seen with the Sony Mavica issue. Releases before 10.4.11 mount disks with Sparing Table but does not read its files correctly. Version 10.4.11 fixes this problem. Similarly,
Windows XP Windows XP is a major release of Microsoft's Windows NT operating system. It was released to manufacturing on August 24, 2001, and later to retail on October 25, 2001. It is a direct upgrade to its predecessors, Windows 2000 for high-end and ...
Service Pack 2 (SP2) cannot read DVD-RW discs that use the UDF 2.00 sparing tables as a defect management system. This problem occurs if the UDF defect management system creates a sparing table that spans more than one sector on the DVD-RW disc. Windows XP SP2 can recognize that a DVD is using UDF, but Windows Explorer displays the contents of a DVD as an empty folder. A hotfix is available for this and is included in Service Pack 3. Due to the default UDF versions and options, a UDF partition formatted by Windows cannot be written under macOS. On the other hand, a partition formatted by macOS cannot be directly written by Windows, due to the requirement of a MBR partition table. In addition, Linux only supports writing to UDF 2.01. A script for Linux and macOS called handles these incompatibilities by using UDF 2.01 and adding a fake MBR; for Windows the best solution is using the command-line tool .


See also

*
Comparison of file systems The following tables compare general and technical information for a number of file systems. General information Limits Metadata Features File capabilities Block capabilities Note that in addition to the below table, blo ...
*
DVD authoring DVD authoring is the process of creating a DVD video capable of playing on a DVD player. DVD authoring software must conform to the specifications set by the DVD Forum. DVD authoring is the second step in the process of producing finished DVD ...
* ISO/IEC 13490


References


Further reading

* ISO/IEC 13346 standard, also known a
ECMA-167


External links


OSTA home page
*UDF specifications
1.021.502.002.012.502.60
(March 1, 2005)
SecureUDFWenguang Wang's UDF IntroductionLinux UDF supportMicrosoft Windows UDF Read Troubleshooting
{{List of International Electrotechnical Commission standards Disk file systems ISO standards IEC standards Ecma standards Windows components